Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make replication flush interval configurable in mysql source #202

Conversation

saurabhojha
Copy link
Contributor

Add a replication flush interval in mysql source config.
This pr closes : #59

@saurabhojha saurabhojha force-pushed the saurabh/transfer/configure-flush-intervals branch from 8b64ab2 to c9109c2 Compare February 6, 2025 08:34
@saurabhojha saurabhojha force-pushed the saurabh/transfer/configure-flush-intervals branch from c9109c2 to 178fab7 Compare February 6, 2025 08:39
Copy link

robot-magpie bot commented Feb 6, 2025

@laskoviymishka has imported your pull request. If you are a member of Transfer team, you can view this diff.

Comment on lines +174 to +175
- **ReplicationFlushInterval** (`time.Duration`): Specifies the replication flush interval. Defined in nanoseconds.
- Example: `5000000000` (5 seconds)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's change it to milliseconds (1/1000 of second). I don't see a case when user will need to specify interval with such accuracy

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see: #215
this would allow to type 5s instead of 5000000000

Copy link
Contributor Author

@saurabhojha saurabhojha Feb 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Migrating to yaml.v3 wouldn't solve this issue. Since we first convert yaml configuration to a TransferYamlView which parses entire config as any type. Converting from json (done later in NewSource and NewDestination methods) to the desired config struct again would raise the following error:

unable to construct missed fields: failed to decode: 1 error(s) decoding:

  • 'ReplicationFlushInterval' expected type 'time.Duration', got unconvertible type 'string', value: '5s'

Copy link

robot-magpie bot commented Feb 17, 2025

✅ This pull request is being closed because it has been successfully merged into our internal monorepository.
Your changes will be pushed to this repository soon. Thank you for your contribution!

@robot-magpie robot-magpie bot closed this Feb 17, 2025
robot-piglet pushed a commit that referenced this pull request Feb 17, 2025
Add a replication flush interval in mysql source config.
This pr closes : #59

---

Pull Request resolved: #202

Co-authored-by: tserakhau <tserakhau@double.cloud>
Co-authored-by: tserakhau <tserakhau@double.cloud>
commit_hash:02ea17e2169b6dac62f1ba31bac43828f96be5ba
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MySQL/source: allow to configure flush-intervals
3 participants